WIP feat: Add new prompts API#2166
Open
Luca Forstner (lforst) wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds an experimental typed prompt API for defining, composing, building, and adapting Braintrust prompts in TypeScript.
Define Prompts
Prompts are defined with typed
inputand optional typedoutputschemas. Both schemas are callback-based, so the schema helpersis scoped to the right context (output schema allows different kind of values than input schema).Message vs String Prompts
renderreturns either a message array orprompt.text.(Note: We need a tagged template for the string prompt, in order to be able to "mustachify" the string)
Prompt Composition
Input schemas can accept already-built prompts or implicitly build specific child prompt definitions.
s.messagesPromptDefinition(...)ands.stringPromptDefinition(...)auto-build a specific child prompt by merging parent input with child input overrides.Use
s.builtMessagesPrompt()ands.builtStringPrompt()when callers must pass an already-built prompt.We track prompt dependencies through message spreads and string interpolation so that we can record all used prompts in traces.
Adapters
Built prompts can be converted to provider-shaped args.
String prompts are coerced to a single user message for message-based adapters.
Safe Extension
.to()always returns an object with.extend(...), which deep-merges additional args while preserving tracing metadata.We do this so that we can always instruct users to pass the prompt as a singular argument to the API, so we can properly preserve
span_info.Not in this PR
For follow-ups, the plan for pushing prompts to Braintrust is something like the following where we "send" a prompt definition to a project: